Incremental collaborative filtering via evolutionary co%2Dclustering.

Collaborative filtering via euclidean embedding.

Detecting Wikipedia vandalism with active learning and statistical language models.
This paper proposes an active learning approach using language model statistics to detect Wikipedia vandalism. Wikipedia is a popular and influential collaborative information system. The collaborative nature of authoring, as well as the high visibility of its content, have exposed Wikipedia articles to vandalism. Vandalism is defined as malicious editing intended to compromise the integrity of the content of articles. Extensive manual efforts are being made to combat vandalism and an automated approach to alleviate the laborious process is needed. This paper builds statistical language models, constructing distributions of words from the revision history of Wikipedia articles. As vandalism often involves the use of unexpected words to draw attention, the fitness (or lack thereof) of a new edit when compared with language models built from previous versions may well indicate that an edit is a vandalism instance. In addition, the paper adopts an active learning model to solve the problem of noisy and incomplete labeling of Wikipedia vandalism. The Wikipedia domain with its revision histories offers a novel context in which to explore the potential of language models in characterizing author intention. As the experimental results presented in the paper demonstrate, these models hold promise for vandalism detection.
A decision support system for cost%2Deffective diagnosis.

Finding Maximal Fully%2DCorrelated Itemsets in Large Databases.

Bagging with Adaptive Costs.
Abstract — Ensemble methods have proved to be highly effective in improving the performance of base learners under most circumstances. In this paper, we propose a new algorithm that combines the merits of some existing techniques, namely bagging, arcing and stacking. The basic structure of the algorithm resembles bagging. However, the misclassification cost of each training point is repeatedly adjusted according to its observed out-of-bag vote margin. In this way, the method gains the advantage of arcing – building the classifier the ensemble needs – without fixating on potentially noisy points. Computational experiments show that this algorithm performs consistently better than bagging and arcing with linear and nonlinear base classifiers. In view of the characteristics of bacing, a hybrid ensemble learning strategy, which combines bagging and different versions of bacing, is proposed and studied empirically. Index Terms — Data mining, ensemble methods Yi Zhang and W. Nick Street, Member, IEEE
Building a hospital referral expert system with a Prediction and Optimization%2DBased Decision Support System algorithm.
11 This study presents a new method for constructing an expert system using a hospital referral problem as an example. Many factors, 12 such as institutional characteristics, patient risks, traveling distance, and chances of survival and complications should be included in the 13 hospital-selection decision. Ideally, each patient should be treated individually, with the decision process including not only their con-14 dition but also their beliefs about trade-offs among the desired hospital features. An expert system can help with this complex decision, 15 especially when numerous factors are to be considered. We propose a new method, called the Prediction and Optimization-Based Deci-16 sion Support System (PODSS) algorithm, which constructs an expert system without an explicit knowledge base. The algorithm obtains 17 knowledge on its own by building machine learning classifiers from a collection of labeled cases. In response to a query, the algorithm 18 gives a customized recommendation, using an optimization step to help the patient maximize the probability of achieving a desired out-19 come. In this case, the recommended hospital is the optimal solution that maximizes the probability of the desired outcome. With proper 20 formulation, this expert system can combine multiple factors to give hospital-selection decision support at the individual level.
Learning to Rank by Maximizing AUC with Linear Programming.
Abstract — Area Under the ROC Curve (AUC) is often used to evaluate ranking performance in binary classification problems. Several researchers have approached AUC optimization by approximating the equivalent Wicoxon-Mann-Whitney (WMW) statistic. We present a linear programming approach similar to 1-norm Support Vector Machines (SVMs) for instance ranking by an approximation to the WMW statistic. Our formulation can be applied to nonlinear problems by using a kernel function. Our ranking algorithm outperforms SVMs in both AUC and classification performance when using RBF kernels, but curiously not with polynomial kernels. We experiment with variations of chunking to handle the quadratic growth of the number of constraints in our formulation.
HACS Heuristic Algorithm for Clustering Subsets.

Learning to Rank by Maximizing AUC with Linear Programming.
Abstract — Area Under the ROC Curve (AUC) is often used to evaluate ranking performance in binary classification problems. Several researchers have approached AUC optimization by approximating the equivalent Wicoxon-Mann-Whitney (WMW) statistic. We present a linear programming approach similar to 1-norm Support Vector Machines (SVMs) for instance ranking by an approximation to the WMW statistic. Our formulation can be applied to nonlinear problems by using a kernel function. Our ranking algorithm outperforms SVMs in both AUC and classification performance when using RBF kernels, but curiously not with polynomial kernels. We experiment with variations of chunking to handle the quadratic growth of the number of constraints in our formulation.


Ensemble Pruning Via Semi%2Ddefinite Programming.
An ensemble is a group of learning models that jointly solve a problem. However, the ensembles generated by existing techniques are sometimes unnecessarily large, which can lead to extra memory usage, computational costs, and occasional decreases in effectiveness. The purpose of ensemble pruning is to search for a good subset of ensemble members that performs as well as, or better than, the original ensemble. This subset selection problem is a combinatorial optimization problem and thus finding the exact optimal solution is computationally prohibitive. Various heuristic methods have been developed to obtain an approximate solution. However, most of the existing heuristics use simple greedy search as the optimization method, which lacks either theoretical or empirical quality guarantees. In this paper, the ensemble subset selection problem is formulated as a quadratic integer programming problem. By applying semi-definite programming (SDP) as a solution technique, we are able to get better approximate solutions. Computational experiments show that this SDP-based pruning algorithm outperforms other heuristics in the literature. Its application in a classifier-sharing study also demonstrates the effectiveness of the method.
Sharing Classifiers among Ensembles from Related Problem Domains.
A classification ensemble is a group of classifiers that all solve the same prediction problem in different ways. It is well-known that combining the predictions of classifiers within the same problem domain using techniques like bagging or boosting often improves the performance. This research shows that sharing classifiers among different but closely related problem domains can also be helpful. In addition, a semi-definite programming based ensemble pruning method is implemented in order to optimize the selection of a subset of classifiers for each problem domain. Computational results on a catalog dataset indicate that the ensembles resulting from sharing classifiers among different product categories generally have larger AUCs than those ensembles trained only on their own categories. The pruning algorithm not only prevents the occasional decrease of effectiveness caused by conflicting concepts among the problem domains, but also provides a better understanding of the problem domains and their relationships. 1.
Bagging with Adaptive Costs.
Abstract — Ensemble methods have proved to be highly effective in improving the performance of base learners under most circumstances. In this paper, we propose a new algorithm that combines the merits of some existing techniques, namely bagging, arcing and stacking. The basic structure of the algorithm resembles bagging. However, the misclassification cost of each training point is repeatedly adjusted according to its observed out-of-bag vote margin. In this way, the method gains the advantage of arcing – building the classifier the ensemble needs – without fixating on potentially noisy points. Computational experiments show that this algorithm performs consistently better than bagging and arcing with linear and nonlinear base classifiers. In view of the characteristics of bacing, a hybrid ensemble learning strategy, which combines bagging and different versions of bacing, is proposed and studied empirically. Index Terms — Data mining, ensemble methods Yi Zhang and W. Nick Street, Member, IEEE
Oblique Multicategory Decision Trees Using Nonlinear Programming.
informs ® doi 10.1287/ijoc.1030.0047 © 2005 INFORMS Induction of decision trees is a popular and effective method for solving classification problems in data-mining applications. This paper presents a new algorithm for multi-category decision tree induction based on nonlinear programming. This algorithm, termed OC-SEP (Oblique Category SEParation), combines the advantages of several other methods and shows improved generalization performance on a collection of real-world data sets.
Customer Targeting A Neural Network Approach Guided by Genetic Algorithms.
informs ® doi 10.1287/mnsc.1040.0296 © 2005 INFORMS One of the key problems in database marketing is the identification and profiling of households that are most likely to be interested in a particular product or service. Principal component analysis (PCA) of customer background information followed by logistic regression analysis of response behavior is commonly used by database marketers. In this paper, we propose a new approach that uses artificial neural networks (ANNs) guided by genetic algorithms (GAs) to target households. We show that the resulting selection rule is more accurate and more parsimonious than the PCA/logit rule when the manager has a clear decision criterion. Under vague decision criteria, the new procedure loses its advantage in interpretability, but is still more accurate than PCA/logit in targeting households. Key words: database marketing; neural networks; genetic algorithms; customer relationship management
An intelligent system for customer targeting a data mining approach.

Cluster%2Ddriven refinement for content%2Dbased digital image retrieval.

Model%2Dbased detection
Computer-aided mammographic prompting systems require the reliable detection of a variety of signs of cancer. In this paper we concentrate on the detection of spiculated lesions in mammograms. A spiculated lesion is typically characterised by an abnormal pattern of linear structures and a central mass. Statistical models have been developed to describe and detect both these aspects of spiculated lesions. We describe a generic method of representing patterns of linear structures, which relies on the use of factor analysis to separate the systematic and random aspects of a class of patterns. We model the appearance of central masses using local scale-orientation signatures based on recursive median filtering, approximated using principal component analysis. For lesions of 16 mm and larger the pattern detection technique results in a sensitivity of 80% at 0.014 false positives per image, whilst the mass detection approach results in a sensitivity 80% at 0.23 false positives per image Simp...
IntelliShopper a proactive%2C personal%2C private shopping assistant.
The IntelliShopper is a shopping assistant designed to empower consumers. It is a personal assistant in that it observes the users while shopping and learns their preferences with respect to various features that characterize shopping items. It is proactive in that it remembers the users&#039; requests and autonomously monitors vendor sites for new items that might match the users&#039; needs and preferences. Finally, it protects users&#039; privacy by means of pseudonymity, IP anonymizing, and trusted filtering. Pseudonymity is achieved through the use of personae; we show that this approach also behooves successful classification. IP anonymizing can be performed in at least two manners, which we discuss and compare in the context of our application. Trusted filtering --- as opposed to merchant-based filtering --- improves privacy by allowing users to select their preferred privacy representative. This paper introduces the IntelliShopper system, discusses its architecture and components, describes a prototype implementation, and outlines preliminary evaluations of its performance.
Adaptive Assistants for Customized E%2DShopping.
Shopping online is still a confusing activity due to many vendors, inconsistent information,  and di#erent vendor site organization. Shopping agents based on user-centered business models  have been unsuccessful in the past because shoppers have not perceived price comparison as  a service worth paying for. As a result shopping bots today are biased in favor of vendors  who pay fees to be listed. To make user-centered business models viable, shopping agents must  become more useful. To this end we describe a shopping assistant capable of learning the  heterogeneous and dynamic preferences of shoppers by observing their behavior unobtrusively,  and of presenting shoppers with adaptively customized information about new items discovered  by autonomously monitoring vendor sites.
Evolutionary model selection in unsupervised learning.
Feature subset selection is important not only for the insight gained from determining relevant modeling variables but also for the improved understandability, scalability, and possibly, accuracy of the resulting models. Feature selection has traditionally been studied in supervised learning situations, with some estimate of accuracy used to evaluate candidate subsets. However, we often cannot apply supervised learning for lack of a training signal. For these cases, we propose a new feature selection approach based on clustering. A number of heuristic criteria can be used to estimate the quality of clusters built from a given feature subset. Rather than combining such criteria, we use ELSA, an evolutionary local selection algorithm that maintains a diverse population of solutions that approximate the Pareto front in a multi-dimensional objective space. Each evolved solution represents a feature subset and a number of clusters; two representative clustering algorithms, K-means and EM, are applied to form the given number of clusters based on the selected features. Experimental results on both real and synthetic data show that the method can consistently find approximate Pareto-optimal solutions through which we can identify the significant features and an appropriate number of clusters. This results in models with better and clearer semantic relevance. 1. 
Incremental feature weight learning and its application to a shape%2Dbased query system.
Similarity between shapes is often measured by computing the distance between two feature vectors. Unfortunately, the feature space cannot always capture the notion of similarity in human perception. So, most current image retrieval systems use weights measuring the importance of each feature. However, the similarity does not vary with equal strength or in the same proportion in all directions in the feature space. In this paper, we present feature weights based on both clustered objects in the database and on relevance feedback. We show that using variance information from shape clusters to guide cluster information for an initial database search gives better results than using the standard Euclidean distance. To automatically incorporate a user&#039;s need, the proposed shape-based query system uses an incremental feature weight learning method that re  nes prototypes. In contrast to existing image database systems, the system can learn from user feedback. Indexing and retrieval results are presented that demonstrate the ecacy of our technique using the well-known Columbia database.
A streaming ensemble algorithm %28SEA%29 for large%2Dscale classification.
Classification
Automatic Feature Mining for Personalized Digital Image Retrieval.

Intelligent Object%2Dbased Image Retrieval Using Cluster%2Ddriven Personal Preference Learning.

Feature selection in unsupervised learning via evolutionary search.
Feature subset selection is an important problem in knowl- edge discovery, not only for the insight gained from deter- mining relevant modeling variables but also for the improved understandability, scalability, and possibly, accuracy of the resulting models. In this paper we consider the problem of feature selection for unsupervised learning. A number of heuristic criteria can be used to estimate the quality of clusters built from a given featuresubset. Rather than combining such criteria, we use ELSA, an evolutionary lo- cal selection algorithm that maintains a diverse population of solutions that approximate the Pareto front in a multi- dimensional objectiv espace. Each evolved solution repre- sents a feature subset and a number of clusters; a standard K-means algorithm is applied to form the given n umber of clusters based on the selected features. Preliminary results on both real and synthetic data show promise in finding Pareto-optimal solutions through which we can identify the significant features and the correct number of clusters.
Efficient and Scalable Pareto Optimization by Evolutionary Local Selection Algorithms.
Local selection is a simple selection scheme in evolutionary computation. Individual fitnesses  are accumulated over time and compared to a fixed threshold, rather than to each  other, to decide who gets to reproduce. Local selection, coupled with fitness functions  stemming from the consumption of finite shared environmental resources, maintains diversity  in a way similar to fitness sharing. However, it is more efficient than fitness sharing  and lends itself to parallel implementations for distributed tasks. While local selection is  not prone to premature convergence, it applies minimal selection pressure to the population.
A Neural Network Model for Prognostic Prediction.
An important and difficult prediction task in many domains, particularly medical decision making, is that of prognosis. Prognosis presents a unique set of problems to a learning system when some of the outputs are unknown. This paper presents a new approach to prognostic prediction, using ideas from nonparametric statistics to fully utilize all of the available information in a neural architecture. The technique is applied to breast cancer prognosis, resulting in flexible, accurate models that may play a role in preventing unnecessary surgeries. 1 Introduction  This paper applies artificial neural network classification to the analysis of survival or lifetime data (Lee, 1992), in which the objective can be broadly defined as predicting the future time of a particular event. In this work we are concerned specifically with prognosis, that is, predicting the course of a disease. These methods are applied to breast cancer prognosis, predicting how long after surgery we can expect the disea...
A Neural Network Model for Prognostic Prediction.
An important and difficult prediction task in many domains, particularly medical decision making, is that of prognosis. Prognosis presents a unique set of problems to a learning system when some of the outputs are unknown. This paper presents a new approach to prognostic prediction, using ideas from nonparametric statistics to fully utilize all of the available information in a neural architecture. The technique is applied to breast cancer prognosis, resulting in flexible, accurate models that may play a role in preventing unnecessary surgeries. 1 Introduction  This paper applies artificial neural network classification to the analysis of survival or lifetime data (Lee, 1992), in which the objective can be broadly defined as predicting the future time of a particular event. In this work we are concerned specifically with prognosis, that is, predicting the course of a disease. These methods are applied to breast cancer prognosis, predicting how long after surgery we can expect the disea...
Feature Selection via Mathematical Programming.
The problem of discriminating between two finite point sets in n-dimensional feature space by a separating plane that utilizes as few of the features as possible, is formulated as a mathematical program with a parametric objective function and linear constraints. The step function that appears in the objective function can be approximated by a sigmoid or by a concave exponential on the nonnegative real line, or it can be treated exactly by considering the equivalent linear program with equilibrium constraints (LPEC). Computational tests of these three approaches on publicly available real-world databases have been carried out and compared with an adaptation of the optimal brain damage (OBD) method for reducing neural network complexity. One feature selection algorithm via concave minimization (FSV) reduced cross-validation error on a cancer prognosis database by 35.4% while reducing problem features from 32 to 4.  Feature selection is an important problem in machine learning [18, 15, 1...
Clustering via Concave Minimization.
The problem of assigning m points in the n-dimensional real space  R  n  to k clusters is formulated as that of determining k centers in  R  n  such that the sum of distances of each point to the nearest center is minimized. If a polyhedral distance is used, the problem can be formulated as that of minimizing a piecewise-linear concave function on a polyhedral set which is shown to be equivalent to a bilinear program: minimizing a bilinear function on a polyhedral set. A fast finite k-Median Algorithm consisting of solving few linear programs in closed form leads to a stationary point of the bilinear program. Computational testing on a number of realworld databases was carried out. On the Wisconsin Diagnostic Breast Cancer (WDBC) database, k-Median training set correctness was comparable to that of the k-Mean Algorithm, however its testing set correctness was better. Additionally, on the Wisconsin Prognostic Breast Cancer (WPBC) database, distinct and clinically important survival curv...
An Inductive Learning Approach to Prognostic Prediction.
This paper introduces the Recurrence Surface Approximation, an inductive learning method based on linear programming that predicts recurrence times using censored training examples, that is, examples in which the available training output may be only a lower bound on the &#034;right answer.&#034; This approach is augmented with a feature selection method that chooses an appropriate feature set within the context of the linear programming generalizer. Computational results in the field of breast cancer prognosis are shown. A straightforward translation of the prediction method to an artificial neural network model is also proposed. 1 INTRODUCTION  Machine learning methods have been successfully applied to the analysis of many different complex problems in recent years, including many biomedical applications. One field which can benefit from this type of approach is the analysis of survival or lifetime data  (Lee, 1992; Miller Jr., 1981), in which the objective can be broadly defined as predicting...
